Excellent piece of framework, this. Intutive enough for even me to understand ;)
I have a query, though:
Is it possible to somehow force phpCrawl to visit certain page(s)? Ie. can I some how add certain pages to the crawl-list?
Ie. if I start with $crawler->setURL("http://www.php.net"); but also in same session want to make sure I visit mysql.com or perhaps a certain sub-section of php.net that wuould not otherwise get crawled? (yes examples are very imaginary obviously ;-)
Thanks in advance :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
Anonymous
-
2020-11-15
Post awaiting moderation.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
Anonymous
-
2021-04-28
Post awaiting moderation.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Excellent piece of framework, this. Intutive enough for even me to understand ;)
I have a query, though:
Is it possible to somehow force phpCrawl to visit certain page(s)? Ie. can I some how add certain pages to the crawl-list?
Ie. if I start with $crawler->setURL("http://www.php.net"); but also in same session want to make sure I visit mysql.com or perhaps a certain sub-section of php.net that wuould not otherwise get crawled? (yes examples are very imaginary obviously ;-)
Thanks in advance :)
Hey,
my examples are WHAT? ;)
And im sorry, there's no setting that let's you add one ore more URL's directly to the queue besides the one in setUrl().
But feel free to add this request to the list of feature-requests: http://sourceforge.net/tracker/?group_id=89439&atid=590149
This little workaround works as well:
Hope i could help
No no - MY examples are very imaginary ;-) I was not talking down about those included :-D
Thanks for your suggestion. Is initChildProcess only a part of multiprocessing approach? I'm not sure I fully understand if/how your example works.
Is there actually an AddUrl function as in your example?
Feature request is added :-)
https://sourceforge.net/tracker/?func=detail&aid=3529802&group_id=89439&atid=590149
And: Thanks for providing such excellent support!
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Any news of this "add url"? Would be quite useful!
Last edit: Anonymous 2014-11-19
View and moderate all "Help" comments posted by this user
Mark all as spam, and block user from posting to "Forum"
Code above works good. Thanks
Last edit: Anonymous 2014-11-22